Skip to content

Comments

Update nuitka to 0.7.6#278

Closed
pyup-bot wants to merge 1 commit intomasterfrom
pyup-update-nuitka-0.5.30-to-0.7.6
Closed

Update nuitka to 0.7.6#278
pyup-bot wants to merge 1 commit intomasterfrom
pyup-update-nuitka-0.5.30-to-0.7.6

Conversation

@pyup-bot
Copy link
Collaborator

This PR updates Nuitka from 0.5.30 to 0.7.6.

Changelog

0.7

********************

This release is massively improving macOS support, esp. for M1 and the
latest OS releases, but it also has massive improvements for usability
and bug fixes in all areas.

Bug Fixes
=========

-  Fix, ``set`` creation wasn't annotating its possible exception exit
from hashing values and is not as free of side effects as ``list``
and ``tuple`` creations are. Fixed in 0.6.19.1 already.

-  Windows: Fix, experimental options got lost for the C compilation
when switching from MSVC to MinGW64, making them have no effect.
Fixed in 0.6.19.1 already.

-  Windows: Fix, Clang from MinGW64 doesn't support LTO at this time,
therefore default to ``no`` for it. Fixed in 0.6.19.1 already.

-  Debian: Fix, failed to detect Debian unstable as suitable for
linking, it doesn't have the release number. Fixed in 0.6.19.1
already.

-  Standalone: Added data files for ``pygsheets`` package. Fixed in
0.6.19.2 already.

-  Fix, paths from plugin related file paths need to be made absolute
before used internally, otherwise the cache can fail to deduplicate
them. Fixed in 0.6.19.2 already.

-  Python3: With gcc before version 5, e.g. on CentOS 7, where we switch
to using ``g++`` instead, the gcc version checks could crash. Fixed
in 0.6.19.2 already.

-  Windows: Disable MinGW64 wildcard expansion for command line
arguments. This was breaking command lines with arguments like
``--filename *.txt``, which under ``cmd.exe`` are left alone by the
shell, and are to be expanded by the program. Fixed in 0.6.19.2
already.

-  Standalone: Added missing implicit dependency needed for
``--follow-stdlib`` with Python for some uses of the ``locale``
module. Fixed in 0.6.19.2 already.

-  Standalone: Added workarounds for newest ``numpy`` that wants to set
``__code__`` objects and required improvements for macOS library
handling. Fixed in 0.6.19.3 already.

-  Windows: Caching of DLL dependencies for the main programs was not
really working, requiring to detect them anew for every standalone
compilation for no good reason. Fixed in 0.6.19.3 already.

-  Windows: Fix, CTRL-C from a terminal was not propagated to child
processes on Windows. Fixed in 0.6.19.4 already.

-  Standalone: With ``certifi`` and Python3.10 the
``importlib.resource`` could trigger Virus scanner inflicted file
access errors. Fixed in 0.6.19.4 already.

-  Python3.10: Reverted error back iteration past end of generator
change for Python 3.10.2 or higher to become compatible with that
too. Fixed in 0.6.19.5 already.

-  Standalone: Added support for ``anyio`` and by proxy for Solana.
Fixed in 0.6.19.5 already.

-  Fix, compilation with resource mode ``incbin`` and ``--debugger`` was
not working together. Fixed in 0.6.19.5 already.

-  Fix, format optimization of known ``str`` objects was not properly
annotating an exception exit when being optimized away, causing
consistency checks to complain. Fixed in 0.6.19.5 already.

-  Windows: Fix, ``clcache`` didn't work for non-standard encoding
source paths due to using th direct mode, where wrong filenames are
output by MSVC. Fixed in 0.6.19.5 already.

-  Windows: Fix, ``ccache`` cannot handle source code paths for
non-standard encoding source paths. Fixed in 0.6.19.5 already.

-  Python2.6: Fix, calls to ``iteritems`` and ``iterkeys`` on known
dictionary values could give wrong values. Fixed in 0.6.19.5 already.

-  Fix, the value of ``__module__`` if set by the metaclass was
overwritten when creating types. Fixed in 0.6.19.6 already.

-  Plugins: Add support for the latest version of ``pkg_resources`` that
has "vendored" even more packages. Fixed in 0.6.19.6 already.

-  Onefile: The onefile binary was locked during run time and could not
be renamed, preventing in-place updates. This has been resolved and
now on Windows, the standard trick for updating a running binary of
renaming it, then placing the new file works.

-  Fix, wasn't checking the ``zstandard`` version and as a result could
crash if too old versions of it. This is now checked.

-  macOS: Large amounts of bug fixes for the dependency scanner. It got
cleaned up and now handles many more cases correctly.

-  Windows: Fix, was not properly detecting wrong ClangCL architecture
mismatch with the Python architecture. This could result in strange
errors during C compilation in this setup.

-  Standalone: Added implicit dependencies for the ``asyncpg`` module.

-  Linux: Detect Debian or Ubuntu base and distribution name more
reliably. This helps esp. with static libpython optimization being
recognized automatically.

New Features
============

-  We now disallow options that take arguments to be provided without
using ``=``.

-  Previously ``--lto no`` worked just as well as ``--lto=no`` did. And
that was the cause of problems when ``--lto`` first became a choice.

Recently similar, but worse problems were observed, where e.g.
``--include-module`` could swallow trailing other arguments when
users forgot to specify the name by accident. Therefore this style of
giving options is now explicitly rejected.

-  Compiled types of Nuitka now inherit from uncompiled types. This
should allow easier and more complete compatibility, making even code
in extension modules that uses ``PyObject_IsInstance`` work, e.g.
``pydanctic``.

-  macOS: Added signing of application bundles and standalone binaries
for deployment to newer macOS platforms and esp. M1 where these are
mandatory for execution.

-  macOS: Added support for selecting the single macOS target arch to
create a binary for. The ``universal`` architecture is not yet
supported though, but will be added in a future release.

-  Added support for compression in onefile mode through the use of an
other Python installation, that has the ``zstandard`` module
installed. With this it will work with 2.6 or higher, but require a
3.5 or higher Python with it installed in either ``PATH`` or on
Windows in the registry alternatively.

-  Added UPX plugin to compress created extension modules and binaries
and for standalone mode, the included DLLs. For onefile, the
compression is not useful since it has the payload already
compressed.

-  Added a more explicit way to list usable MSVC versions with
``--msvc=list`` rather than requiring an invalid value. Check values
given in the same way that Scons will do.

-  Added support for ``--python-flag=-u`` which disabled outputs
buffers, so that these outputs are written immediately.

-  Plugins: Always on plugins now can have command line options. We want
this for the ``anti-bloat`` plugin that is enabled by default in this
release.

-  Plugins: Added ability for plugin to provide fake dependencies for a
module. We want the this for the ``multiprocessing`` plugin, that is
now enabled by default in this release too.

-  Plugins: Added ability for plugins to modify DLLs after copy for
standalone. We will be using this in the new ``upx`` plugin.

-  Added retry for file copies that fail due to still running program.
This can happen on Windows with DLLs in standalone mode. For
interactive compilation, this allows a retry to happen after
prompting the user.

-  UI: Added ability to list MSVC versions with ``--msvc=list``, and
detect illegal values given to ``--msvc=`` before Scons sees them, it
also crashes with a relative unhelpful error message.

-  UI: When linking, close the C compilation progress bar and state that
that linking is going on. For some very large LTO compilations, it
was otherwise at 100% and still taking a long time, confusing users.

-  Plugins: Added new plugin that is designed to handle DLL dependencies
through a configuration file that can both handle filename patterns
as well as code provided DLL locations.

-  Optimization: Exclude parts of the standard library by default. This
allows for much smaller standalone distributions on modules, that can
be expected to never be an implicit dependency of anything, e.g.
``argparse`` or ``pydoc``.

Optimization
============

-  Standalone: Do not include ``encodings.bz2_codec`` and
``encodings.idna`` anymore, these are not file system encodings, but
require extension modules.

-  Make sure we use proper ``(void)`` arguments for C functions without
arguments, as for C functions, that makes a real difference, they are
variable args functions and more expensive to call otherwise.

-  For standalone, default to using ``--python-flag=no_site`` to avoid
the overhead that the typically unused ``site`` module incurs. It
often includes large parts of the standard library, which we now want
to be more selective about. There is new Python flag added called
``--python-flag=site`` that restores the inclusion of ``site``
module.

-  Standalone: Exclude non-critical codec modules from being technical,
i.e. have to be available at program startup. This removes the need
for e.g. ``bz2`` related extension modules previously included.

-  In reformulations, use dictionary methods directly, we have since
introduced dictionary specific methods, and avoid the unnecessary
churn during optimization.

-  The complex call helper could trigger unnecessary passes in some
cases. The pure functions were immediately optimized, but usages in
other modules inside loops sometimes left them in incomplete states.

-  Windows: Avoid repeated hashing of the same files over and over for
``clcache``.

-  Cache dependencies of bytecode demoted modules in first compile and
reuse that information in subsequent compilations.

-  Linux: Added option for switching compression method for onefile
created with ``AppImage``. The default is also now ``gzip`` and not
``xz`` which has been observed to cause much slower startup for
little size gains.

-  Standalone: For failed relative imports, during compiled time
absolute imports were attempted still and included if successful, the
imports would not be use them at run time, but lead to more modules
being included than necessary.

Organisational
==============

-  There is now a `Discord server for Nuitka community
<https://discord.gg/nZ9hr9tUck>`__ where you can hang out with the
developers and ask questions. It is mirrored with the Gitter
community chat, but offers more features.

-  The ``anti-bloat`` is now on by default. It helps scalability by
changing popular packages to not provide test frameworks,
installation tools etc. in the resulting binary. This oftentimes
reduces the compilation by thousands of modules.

-  Also the ``multiprocessing`` plugin is now on by default. Detecting
its need automatically removes a source of problems for first time
users, that didn't know to enable it, but had all kinds of strange
crashes from multiprocessing malfunctioning. This should enhance the
out of the box experience by a lot.

-  With this release, the version numbering scheme will be changed. For
a long time we have used 4 digits, where one is a leading zero. That
was initially done to indicate that it's not yet ready. However, that
is just untrue these days. Therefore, we switch to 3 digits, and a
first hotfix with now be 0.7.1 rather than 0.6.19.1, which is too
long.

It has been observed that people disregard differences in the third
digit, but actually for Nuitka these have oftentimes been very
important updates. This change is to rectify it, and a new release
will be ``0.8``, and there will be a ``1.0`` release after ``0.9``.

-  Added a new section to User Manual that explains how to manually load
files, such that it is cleaner and compatible code. Using paths
relative to current directory is not the right way, but there are
nice helpers that make it very simple and correct with all kinds of
contexts.

-  Report the MSVC version in Scons output during compilation. The 2022
version is required, but we support everything back to 2008, to work
on very old systems as well. This will help identifying differences
that arise from there.

-  Quality: Find Clang format from MSVC 2022 too. We use in auto format
of Nuitka source code, but need to also search that as a new path.

-  Added a spellchecker extension for Visual Code, resulting in many
spelling fixes in all kinds of documentation and code. This finds
more things than ``codespell``, but also has a lot of false alarms.

-  Check value of ``--onefile-tempdir-spec`` for typical user errors. It
cannot be ``.`` as that would require to overwrite the onefile binary
on Windows, and will generally behave very confusing. Warn about
absolute or relative paths going outside of where the binary lives.
Can be useful in controlled setups, but not generally. Also warn
about using no variables, making non-unique paths.

-  macOS: Flavor detection was largely expanded. The ``Apple`` flavor is
recognized on more systems. ``Homebrew`` was newly added, and we
actually can detect ``CPython`` reliably as a first.

-  Added a tool from leo project to create better ``.pyi`` files for
modules. We will make use of it in the future to enhance the files
created by Nuitka to not only contain hidden dependencies, but
optionally also module signatures.

-  Plugins: Clearer information from ``pyside2`` that patched wheels
might be mandatory and workarounds only patches cannot be done for
older Python.

-  Added progress bars for DLL dependency detection and DLL copying for
standalone. These both can end up using take a fair bit of time
depending on project size, and it's nice to know what's going on.

-  macOS: Added support for using both ``--onefile`` and
``--macos-create-app-bundle`` as it is needed for PySide2 due to
issues with signing code now.

-  Added warning when attempting to include extension modules in an
accelerated compilation.

-  Modules: Catch the user error of following all imports when creating
a module. This is very unlikely to produce usable results.

-  Start integrating `Sourcery <https://sourcery.ai>`__ for improved
Nuitka code. It will comment on PRs and automatically improve Nuitka
code as we develop it.

-  Debugging: Added command line tool ``find-module`` that outputs how
Nuitka locates a module in the Python environment it's ran with. That
removes the need to use Python prompt to dump ``__file__`` of
imported modules. Some modules even hide parts of their namespace
actively during run-time, the tool will not be affected by that.

Cleanups
========

-  Refactored Python scan previously used for Scons usage on versions
that need to run in with another Python to be more generally usable.

-  Use explicit ``nuitka.utils.Hashing`` module that allows the core to
perform these operations with simpler code.

-  macOS: Use ``isPathBelow`` for checking if something is a system
library for enhanced robustness and code clarity.

-  macOS: Make sure to use our proper error checking wrappers for
command execution when using tools like ``otool`` or ``codesign``.

-  Standalone: Avoid a temporary file with a script during technical
import detection. These have been observed to potentially become
corrupted, and this avoids any chance of that happening, while also
being simpler code.

-  Avoid naming things ``shlib`` and call them ``extension`` instead.
Inspired by the spell checker disliking the former term, which is
also less precise.

-  Removed the dead architecture selection option for Windows, it was
unused for a long time.

-  Moved Windows ``SxS`` handling of DLLs to a more general place where
also macOS specific tasks are applied, to host standard modification
of DLLs during their copying.

Tests
=====

-  Better matching of relative filenames for search modes of the
individual test suite runners.

-  Debugger outputs on segfaults were no longer visible and have been
restored.

Summary
=======

This release is tremendous progress for macOS. Finally biting the bullet
and paying obscene amounts of money to rent an M1 machine, it was
possible to enhance the support for this platform. Currently typical
packages for macOS are being made compatible as well, so it can now be
expected to perform equally well.

On the quality side, the spell checker has had some positive effects,
finding typos and generally misspelled code, that ``codespell`` does
not, due to it being very conservative.

The trend to enhance plugins has continued. The copying of DLLs is very
nearly finalized. Making more plugins enabled by default is seeing a lot
of progress, with 2 important ones addressed.

Work on the size of distributions has seen a lot of positive results, in
that now standalone distributions are often very minimal, with many
extension modules from standard library no longer being present.

***********************

0.6.19

***********************

This release adds support for 3.10 while also adding very many new
optimization, and doing a lot of bug fixes.

Bug Fixes
=========

-  Calls to ``importlib.import_module`` with expressions that need
releases, i.e. are not constant values, could crash the compilation.
Fixed in 0.6.18.1 already.

-  After a fix for the previous release, modules that fail to import are
attempted again when another import is executed. However, during this
initialization for top level module in ``--module`` mode, this was
was done repeatedly, and could cause issues. Fixed in 0.6.18.1
already.

-  Standalone: Ignore warning given by ``patchelf`` on Linux with at
least newer OpenSUSE. Fixed in 0.6.18.1 already.

-  Fix, need to avoid computing large values out of ``<<`` operation as
well. Fixed in 0.6.18.2 already.

.. code:: python

    This large value was computed at runtime and then if used, also
    converted to a string and potentially hashed, taking a long time.
   1 << sys.maxint

-  Standalone: Ignore warning given by ``patchelf`` on Linux about a
workaround being applied.

-  Fix, calls to ``importlib.import_module`` were not correctly creating
code for dynamic argument values that need to be released, causing
the compilation to report the error. Fixed in 0.6.18.1 already.

-  MSYS2: Fix, the console scripts are actually good for it as opposed
to CPython, and the batch scripts should not be installed. Fixed in
0.6.18.2 already.

-  Setuptools: Added support older version of ``setuptools`` in meta
``build`` integration of Nuitka.

-  Fix, calls to ``importlib.import_module`` with 2 arguments that are
dynamic, were not working at all. Fixed in 0.6.18.2 already.

-  Windows: Compiling with MinGW64 without ``ccache`` was not working
due to issues in Scons. Fixed in 0.6.18.2 already.

-  Fix, the ``repr`` built-in was falsely annotated as producing a
``str`` value, but it can be also derived or ``unicode`` in Python2.

-  Fix, attribute nodes were not considering the value they are looking
up on. Now that more values will know to have the attributes, that
was causing errors. Fixed in 0.6.18.2 already.

-  Fix, left shifting can also produce large values and needs to be
avoided in that case, similar to what we do for multiplications
already. Fixed in 0.6.18.2 already.

-  UI: The new option ``--disable-ccache`` didn't really have the
intended effect. Fixed in 0.6.18.3 already.

-  UI: The progress bar was causing tearing and corrupted outputs, when
outputs were made, now using proper ``tqdm`` API for doing it, this
has been solved. Fixed in 0.6.18.4 already.

-  Fix, the constant value ``sys.version_info`` didn't yet have support
for its type to be also a compile time constant in e.g. tuples. Fixed
in 0.6.18.4 already.

-  Onefile: Assertions were not disabled, and on Windows with MinGW64
this lead to including the C filenames of the ``zstd`` inline copy
files and obviously less optimal code. Fixed in 0.6.18.4 already.

-  Standalone: Added support for ``bottle.ext`` loading extensions to
resolve at compile time. Fixed in 0.6.18.5 already.

-  Standalone: Added support for ``seedir`` required data file. Fixed in
0.6.18.5 already.

-  MSYS2: Failed to link when using the static libpython, which is also
now the default for MSYS2. Fixed in 0.6.18.5 already.

-  Python3.6+: Fix, the intended finalizer of compiled ``asyncgen`` was
not present and in fact associated to help type. This could have
caused corruption, but that was also very unlikely. Fixed in 0.6.18.5
already.

-  Python3: Fix, need to set ``__file__`` before executing modules, as
some modules, e.g. newer PyWin32 use them to locate things during
their initialization already.

-  Standalone: Handle all PyWin32 modules that need the special DLLs and
not just a few.

-  Fix, some ``.pth`` files create module namespaces with ``__path__``
that does not exist, ignore these in module importing.

-  Python2.6-3.4: Fix, modules with an error could use their module name
after it was released.

-  Distutils: When providing arguments, the method suggested in the docs
is not compatible with all other systems, e.g. not
``setuptools_rust`` for which a two elemented tuple form needs to be
used for values. Added support for that and documented its use as
well in the User Manual.

-  Python3.7+: Do no longer allow deleting cell values, this can lead to
corruption and should be avoided, it seems unlikely outside of tests
anyway.

-  Standalone: Added support for more ciphers and hashes with
``pycryptodome`` and ``pycryptodomex``, while also only including
Ciphers when needed.

-  Distutils: Was not including modules or packages only referenced in
the entry point definition, but not in the list of packages. That is
not compatible and has been fixed.

-  Fix, must not expose the constants blob from extension modules, as
loading these into a compiled binary can cause issues in this case.

-  Standalone: Added support for including OpenGL and SSL libraries with
``PySide2`` and ``PySide6`` packages.

-  Windows: Fix, the ``cmd`` files created for uninstalled Python and
accelerated programs to find the Python installation were not passing
command line arguments.

-  Windows: Executing modules with ``--run`` was not working properly
due to missing escaping of file paths.

-  Fix, parsing ``.pyi`` files that make relative imports was not
resolving them correctly.

-  Python3: Fix, when disabling the console on Windows, make sure the
file handles still work and are not ``None``.

-  Windows: Fix, need to claim all OS versions of Windows as supported,
otherwise e.g. high DPI features are not available.

New Features
============

-  Programs that are to be executed with the ``-m`` flag, can now be
compiled with ``--python-flag=-m`` and will then behave in a
compatible way, i.e. load the containing package first, and have a
proper ``__package__`` value at run time.

-  We now can write XML reports with information about the compilation.
This is initially for use in PGO tests, to decide if the expected
forms of inclusions have happened and should grow into a proper
reporting tool over time. At this point, the report is not very
useful yet.

-  Added support for Python 3.10, only ``match`` statements are not
completely supported. Variations with ``|`` matches that also assign
are not allowed currently.

-  Windows: Allow using ``--clang`` with ``--mingw64`` to e.g. use the
``clang.exe`` that is contained in the Nuitka automatic download
rather than ``gcc.exe``.

-  Added support for Kivy. Works through a plugin that is automatically
enabled and needs no other inputs, detecting everything from using
Kivy at compile time.

-  Added initial support for Haiku OS, a clone of BeOS with a few
differences in their Python installation.

-  Added experimental plugin ``trio`` that works around issues with that
package.

Optimization
============

-  Also trust hard imports made on the module level in function level
code, this unlocks many more static optimization e.g. with
``sys.version_info`` when the import and the use are not on the same
level.

-  For the built-in type method calls with generic implementation, we
now do faster method descriptor calls. These avoid creating a
temporary ``PyCFunction`` object, that the normal call slot would,
this should make these calls faster. Checking them for compiled
function, etc. was only wasteful, so this makes it more direct.

-  Loop and normal merge traces were keeping assignments made before the
loop or inside a branch, that was otherwise unused alive. This should
enable more optimization for code with branches and loops. Also
unused loop traces are now recognized and removed as well.

-  Avoiding merges of escaped traces with the unescaped trace, there is
no point in them. This was actually happening a lot and should mean a
scalability improvement and unlock new optimization as well.

-  Avoid escaping un-init traces. Unset values need not be considered as
potentially modified as that cannot be done.

-  The ``str`` shape is now detected through variables, this enables
many optimization on the function level.

-  Added many ``str`` operation nodes.

These are specifically all methods with no arguments, as these are
very generic to add, introduced a base class for them, where we know
they all have no effect or raise, as these functions are all
guaranteed to succeed and can be served by a common base class.

This covers the ``str.capitalize``, ``str.upper``, ``str.lower``,
``str.swapcase``, ``str.title``, ``str.isalnum``, ``str.isalpha``,
``str.isdigit``, ``str.islower``, ``str.isupper``, ``str.isspace``,
and ``str.istitle`` functions.

For static optimization ``str.find`` and ``str.rfind`` were added, as
they are e.g. used in a ``sys.version.find(...)`` style in the ``os``
module, helping to decide to not consider ``OS/2`` only modules.

Then, support for ``str.index`` and ``str.rindex`` was added, as
these are very similar to ``str.find`` forms, only that these may
raise an exception.

Also add support for ``str.split`` and ``str.rsplit`` which will be
used sometimes for code needed to be compile time computed, to e.g.
detect imports.

Same goes for ``endswith`` and ``startswith``, the later is e.g.
popular with ``sys.platform`` checks, and can remove a lot of code
from compilation with them now being decided at compile time.

.. note::

   A few ``str`` methods are still missing, with time we will achieve
   all of them, but this will take time.

-  Added trust for ``sys.builtin_module_names`` as well. The ``os``
module is using it to make platform determinations.

-  When writing constant values, esp. ``tuple``, ``list``, or ``dict``
values, an encoding of "last value" has been added, avoiding the need
to repeat the same value again, making many values more compact.

-  When starting Nuitka, it usually restarts itself with information
collected in a mode without the ``site`` module loaded, and with hash
randomization disabled, for deterministic behaviour. There is a
option to prevent this from happening, where the goal is to avoid it,
e.g. in testing, say for the coverage taking, but that meant to parse
the options twice, which also loads a lot of code.

Now only a minimal amount of code is used, and the options are parsed
only on the restart, and then an error is raised when it notices, it
was not allowed to do so. This also makes code a lot cleaner.

-  Specialized comparison code for Python2 ``long`` and Python3 ``int``
code, making these operations much faster to use.

-  Specialized comparison code for Python2 ``unicode`` and Python3
``str`` code, making these operations much faster to use, currently
only ``==`` and ``!=`` are fully accelerated, the other comparisons
will follow.

-  Enable static libpython with Python3 Debian packages too. As with
Python2, this will improve the performance of the created binary a
lot and reduce size for standalone distribution.

-  Comparisons with ``in`` and ``not in`` also consider value traces and
go through variables as well where possible. So far only the rich
comparisons and ``is`` and ``is not`` did that.

-  Create fixed import nodes in re-formulations rather than
``__import__`` nodes, avoiding later optimization doing that, and of
course that's simpler code too.

-  Python 3.10: Added support for ``union`` types as compiled time
constants.

-  Modules are now fully optimized before considering which modules they
are in turn using, this avoids temporary dependencies, that later
turn out unused, and can shorten the compilation in some cases by a
lot of time.

-  On platforms without a static link library, in LTO mode, and with
gcc, we can use the ``-O3`` mode, which doesn't work for
``libpython``, but that's not used there. This also includes fake
static libpython, as used by MinGW64 and Anaconda on Windows.

-  The ``anti-bloat`` plugin now also handles newer ``sklearn`` and
knows more about the standard library, and its runners which it will
exclude from compilation if use for it. Currently that is not the
default, but it should become that.

Organisational
==============

-  Migrated the Nuitka blog from Nikola to Sphinx based ABlog and made
the whole site render with Sphinx, making it a lot more usable.

-  Added a small presentation about Nuitka on the Download page, to make
sure people are aware of core features.

-  The ``gi`` plugin is now always on. The copying of the ``typelib``
when ``gi`` is imported is harmless and people can disable the plugin
if that's not needed.

-  The ``matplotlib`` plugin is new and also always on. It previously
was part of the ``numpy`` plugin, which is doing too many unrelated
things. Moving this one out is part of a plan to split it up and have
it on by default without causing issues.

-  MSYS2: Detecting ``MinGW`` and ``POSIX`` flavors of this Python. For
the ``MinGW`` flavor of MSYS2, the option ``--mingw64`` is now the
default, before it could attempt to use MSVC, which is not going to
work for it. And also the Tcl and Tk installations of it are being
detected automatically for the ``tk-inter`` plugin.

-  Added Windows version to Nuitka version output, so we have this for
bug reports.

-  User Manual: Added example explaining how to access values from your
code in Nuitka project options.

-  UI: For Python flavors where we expect a static libpython, the error
message will now point out how to achieve it for each flavor.

-  UI: Disable progress bar when ``--show-scons`` is used, it makes
capturing the output from the terminal only harder.

-  UI: Catch error of specifying both ``--msvc=`` and ``--mingw64``
options.

-  Distutils: Improved error messages when using ``setuptools`` or
``build`` integration and failing to provide packages to compile.

-  Plugins: Removed now unused feature to rename modules on import, as
it was only making the code more complex, while being no more needed
after recently adding a place for meta path based importers to be
accounted for.

-  Twitter: Use embedded Tweet in Credits, and regular follow button in
User Manual.

-  Warnings about imports not done, are now only given when optimization
can not remove the usage, and no options related to following have
been given.

-  Added Windows version to ``--version`` output of Nuitka. This is to
more clearly recognize Windows 10 from Windows 11 report, and also
the odd Windows 7 report, where tool chain will be different.

-  In Visual Code, the default Python used is now 3.9 in the "Linux" C
configuration. This matches Debian Bullseye.

-  Nicer outputs from check mode of the auto-format as run for CI
testing, displays problematic files more clearly.

-  Remove broken links to old bug tracker that is no longer online from
the Changelog.

-  UI: When hitting CTRL-C during initial technical import detection, no
longer ask to submit a bug report with the exception stack, instead
exit cleanly.

-  Windows: Enable LTO mode for MinGW64 and other gcc by default. We
require a version that can do it, so take advantage of that.

-  For cases, where code generation of a module takes long, make sure
its name is output when CTRL-C is hit.

-  Windows: Splash screen only works with MSVC, added error indicator
for MinGW64 that states that and asks for porting help.

Cleanups
========

-  Generate all existing C code for generic builtin type method calls
automatically, and use those for method attribute lookups, making it
easier to add more.

-  Changed ``TkInter`` module to data file providing interface, yielding
the 2 directories in question, with a filter for ``demos``.

-  The importing code got a major overhaul and no longer works with
relative filenames, or filenames combined with package names, and
module names, but always only with module names and absolute
filenames. This cleans up some of the oldest and most complex code in
Nuitka, that had grown to address various requirements discovered
over time.

-  Major cleanup of Jinja2 template organisation.

Renamed all C templates from ``.j2`` to ``.c.j2`` for clarity, this
was not done fully consistent before. Also move all C templates to
``nuitka.codegen`` package data, it will be confusing to make a
difference between ones used during compile time and for the static
generation, and the lines are going to become blurry.

Added Jinja2 new macro ``CHECK_OBJECTS`` to avoid branches on
argument count in the call code templates. More of these things
should be added.

Cleanup of code that generates header declarations, there was some
duplication going on, that made it hard to generate consistent code.

-  Removed ``nuitka.finalizatios.FinalizationBase``, we only have one
final visitor that does everything, and that of course makes a lot of
sense for its performance.

-  Major cleanup of the Scons C compiler configuration setup. Moved
things to the dedicate function, and harmonized it more.

-  Resolved deprecation warnings given by with ``--python-debug`` for
Nuitka.

Tests
=====

-  Started test suite for Python PGO, not yet completely working though,
it's not yet doing what is needed though.

-  Added generated test that exercises str methods in multiple
variations.

-  Revived ``reflected`` test suite, that had been removed, because of
Nuitka special needs. This one is not yet passing again though, due
to a few details not yet being as compatible as needed.

-  Added test suite for CPython 3.10 and enable execution of tests with
this version on Github actions.

Summary
=======

This release is another big step forward.

The amount of optimization added is again very large, some of which yet
again unlocks more static optimization of module imports, that
previously would have to be considered implicit. Now analysing these on
the function level as well, we can start searching for cases, where it
could be done, but is not done yet.

After starting with ``dict``, method optimization has focused on ``str``
which is esp. important for static optimization of imports. The next
goal will here be to cover ``list`` which are important for run time
performance and currently not yet optimized. Future releases will
progress there, and also add more types.

The C type specialization for Python3 has finally progressed, such that
it is also covering the ``long`` and ``unicode`` and as such not limited
to Python2 as much. The focus now needs to turn back to not working with
``PyObject *`` for these types, but e.g. with ``+= 1`` to make it
directly work with ``CLONG`` rather than ``LONG`` for which structural
changes in code generation will be needed.

For scalability, the ``anti-bloat`` work has not yet progressed as much
as to be able to enable it by default. It needs to be more possible to
disable it where it causes problems, e.g. when somebody really wants to
include ``pytest`` and test frameworks generally, that's something that
needs to be doable. Compiling without ``anti-bloat`` plugin is something
that is immediately noticeable in exploding module amounts. It is very
urgently recommended to enable it for your compilations.

The support for Windows has been further refined, actually fixing a few
important issues, esp. for the Qt bindings too.

This release adds support for 3.10 outside of very special ``match``
statements, bringing Nuitka back to where it works great with recent
Python. Unfortunately ``orderedset`` is not available for it yet, which
means it will be slower than 3.9 during compilation.

Overall, Nuitka is closing many open lines of action with this. The
``setuptools`` support has yet again improved and at this point should
be very good.

***********************

0.6.18

***********************

This release has a focus on new features of all kinds, and then also new
kinds of performance improvements, some of which enable static
optimization of what normally would be dynamic imports, while also
polishing plugins and adding also many new features and a huge amount of
organisational changes.

Bug Fixes
=========

-  Python3.6+: Fixes to asyncgen, need to raise ``StopAsyncInteration``
rather than ``StopIteration`` in some situations to be fully
compatible.

-  Onefile: Fix, LTO mode was always enabled for onefile compilation,
but not all compilers support it yet, e.g. MinGW64 did not. Fixed in
0.6.17.1 already.

-  Fix, ``type`` calls with 3 arguments didn't annotate their potential
exception exit. Fixed in 0.6.17.2 already.

-  Fix, trusted module constants were not working properly in all cases.
Fixed in 0.6.17.2 already.

-  Fix, ``pkg-resources`` exiting with error at compile time for
unresolved requirements in compiled code, but these can of course
still be optional, i.e. that code would never run. Instead give only
a warning, and runtime fail on these. Fixed in 0.6.17.2 already.

-  Standalone: Prevent the inclusion of ``drm`` libraries on Linux, they
have to come from the target OS at runtime. Fixed in 0.6.17.2
already.

-  Standalone: Added missing implicit dependency for ``ipcqueue``
module. Fixed in 0.6.17.3 already.

-  Fix, Qt webengine support for everything but ``PySide2`` wasn't
working properly. Partially fixed in 0.6.17.3 already.

-  Windows: Fix, bootstrap splash screen code for Windows was missing in
release packages. Fixed in 0.6.17.3 already.

-  Fix, could crash on known implicit data directories not present.
Fixed in 0.6.17.3 already.

-  macOS: Disable download of ``ccache`` binary for M1 architecture and
systems before macOS 10.14 as it doesn't work on these. Fixed in
0.6.17.3 already.

-  Standalone: The ``pendulum.locals`` handling for Python 3.6 was
regressed. Fixed in 0.6.17.4 already.

-  Onefile: Make sure the child process is cleaned up even after its
successful exit. Fixed in 0.6.17.4 already.

-  Standalone: Added support for ``xmlschema``. Fixed in 0.6.17.4
already.

-  Standalone: Added support for ``curses`` on Windows. Fixed in
0.6.17.4 already.

-  Standalone: Added support for ``coincurve`` module. Fixed in 0.6.17.5
already.

-  Python3.4+: Up until Python3.7 inclusive, a workaround for stream
encoding (was ASCII), causing crashes on output of non-ASCII, other
Python versions are not affected. Fixed in 0.6.17.5 already.

-  Python2: Workaround for LTO error messages from older gcc versions.
Fixed in 0.6.17.5 already.

-  Standalone: Added support for ``win32print``. Fixed in 0.6.17.6
already.

-  Fix, need to prevent usage of static ``libpython`` in module mode or
else on some Python versions, linker errors can happen. Fixed in
0.6.17.6 already.

-  Standalone: Do not load ``site`` module early anymore. This might
have caused issues in some configurations, but really only would be
needed for loading ``inspect`` which doesn`t depend on it in
standalone mode. Fixed in 0.6.17.6 already.

-  Fix, could crash with generator expressions in finally blocks of
tried blocks that return. Fixed in 0.6.17.7 already.

.. code:: python

   try:
      return 9
   finally:
      "".join(x for x in b"some_iterable")

-  Python3.5+: Compatibility of comparisons with ``types.CoroutineType``
and ``types.AsyncGeneratorType`` types was not yet implemented. Fixed
in 0.6.17.7 already.

.. code:: python

    These already worked:
   assert isinstance(compiledCoroutine(), types.CoroutineType) is True
   assert isinstance(compiledAsyncgen(), types.AsyncGeneratorType) is True

    These now work too:
   assert type(compiledCoroutine()) == types.CoroutineType
   assert type(compiledAsyncgen()) == types.AsyncGeneratorType

-  Standalone: Added support for ``ruamel.yaml``. Fixed in 0.6.17.7
already.

-  Distutils: Fix, when building more than one package, things could go
wrong. Fixed in 0.6.17.7 already.

-  Fix, for module mode filenames are used, and for packages, you can
specify a directory, however, a trailing slash was not working. Fixed
in 0.6.17.7 already.

-  Compatibility: Fix, when locating modules, a package directory and an
extension module of the same name were not used according to
priority. Fixed in 0.6.17.7 already.

-  Standalone: Added workaround ``importlib_resources`` insisting on
Python source files to exist to be able to load datafiles. Fixed in
0.6.17.7 already.

-  Standalone: Properly detect usage of hard imports from standard
library in ``--follow-stdlib`` mode.

-  Standalone: Added data files for ``opensapi_spec_validator``.

-  MSYS2: Fix, need to normalize compiler paths before comparing.

-  Anaconda: For accelerated binaries, the created ``.cmd`` file wasn't
containing all needed environment.

-  macOS: Set minimum OS version derived from the Python executable
used, this should make it work on all supported platforms (of that
Python).

-  Standalone: Added support for automatic inclusion of ``xmlschema``
package datafiles.

-  Standalone: Added support for automatic inclusion of ``eel`` package
datafiles.

-  Standalone: Added support for ``h5py`` package.

-  Standalone: Added support for ``phonenumbers`` package.

-  Standalone: Added support for ``feedparser`` package, this currently
depends on the ``anti-bloat`` plugin to be enabled, which will become
enabled by default in the future.

-  Standalone: Added ``gi`` plugin for said package that copies
``typelib`` files and sets the search path for them in standalone
mode.

-  Standalone: Added necessary datafiles for ``eel`` package.

-  Standalone: Added support for ``QtWebEngine`` to all Qt bindings and
also make it work on Linux. Before only PySide2 on Windows was
supported.

-  Python3: Fix, the ``all`` built-in was wrongly assuming that bytes
values could not be false, but in fact they are if they contain
``\0`` which is actually false. The same does not happen for string
values, but that's a difference to be considered.

-  Windows: The LTO was supposed to be used automatically on with MSVC
14.2 or higher, but that was regressed and has been repaired now.

-  Standalone: Extension modules contained in packages, depending on
their mode of loading had the ``__package__`` value set to a wrong
value, which at least impacted new matplotlib detection of Qt
backend.

-  Windows: The ``python setup.py install`` was installing binaries for
no good reason.

New Features
============

-  Setuptools support. Documented ``bdist_nuitka`` and ``bdist_wheel``
integration and added support for Nuitka as a ``build`` package
backend in ``pyproject.toml`` files. Using Nuitka to build your
wheels is supposed to be easy now.

-  Added experimental support for Python 3.10, there are however still
important issues with compatibility with the CPython 3.9 test suite
with at least asyncgen and coroutines.

-  macOS: For app bundles, version information can be provided with the
new option ``--macos-app-version``.

-  Added Python vendor detection of ``Anaconda``, ``pyenv``, ``Apple
Python``, and ``pyenv`` and output the result in version output, this
should make it easiert to analyse reported issues.

-  Plugins: Also handle the usage of ``__name__`` for metadata version
resolution of the ``pkg-resources`` standard plugin.

-  Plugins: The ``data-files`` standard plugin now reads configuration
from a Yaml file that ``data-files.yml`` making it more accessible
for contributions.

-  Windows: Allow enforcing usage of MSVC with ``--msvc=latest``. This
allows you to prevent accidental usage of MinGW64 on Windows, when
MSVC is intended, but achieves that without fixing the version to
use.

-  Windows: Added support for LTO with MinGW64 on Windows, this was
previously limited to the MSVC compiler only.

-  Windows: Added support for using ``--debugger`` with the downloaded
MinGW64 provided ``gdb.exe``.

.. note::

   It doesn`t work when executed from a Git bash prompt, but e.g.
   from a standard command prompt.

-  Added new experimental flag for compiled types to inherit from
uncompiled types. This should allow easier and more complete
compatibility, making even code in extension modules that uses
``PyObject_IsInstance`` work, providing support for packages like
``pydanctic``.

-  Plugins: The Qt binding plugins now resolve ``pyqtgraph`` selection
of binding by hard coding ``QT_LIB``. This will allow to resolve its
own dynamic imports depending on that variable at compile time. At
this time, the compile time analysis is not covering all cases yet,
but we hope to get there.

-  macOS: Provide ``minOS`` for standalone builds, derived from the
setting of the Python used to create it.

-  UI: Added new option ``--disable-ccache`` to prevent Nuitka from
injecting ``ccache`` (Clang, gcc) and ``clcache`` (MSVC) for caching
the C results of the compilation.

-  Plugins: Added experimental support for ``PyQt6``. While using
``PySide2`` or ``PySide6`` is very much recommended with Nuitka, this
allows its use.

-  UI: Added option ``--low-memory`` to allow the user to specify that
the compilation should attempt to use less memory where possible,
this increases compile times, but might enable compilation on some
weaker machines.

Optimization
============

-  Added dedicated attribute nodes for attribute values that match names
of dictionary operations. These are optimized into dedicate nodes for
methods of dictionaries should their expression have an exact
dictionary shape. These in turn optimize calls on them statically
into dictionary operations. This is done for all methods of ``dict``
for both Python2 and Python3, namely ``get``, ``items``,
``iteritems``, ``itervalues``, ``iterkeys``, ``viewvalues``,
``viewkeys``, ``pop``, ``setdefault``, ``has_key``, ``clear``,
``copy``, ``update``.

The new operation nodes also add compile time optimization for being
used on constant values where possible.

-  Also added dedicated attribute nodes for string operations. For
operations, currently only part of the methods are done. These are
currently only ``join``, ``strip``, ``lstrip``, ``rstrip``,
``partition``, ``rpartition``. Besides performance, this subset was
enough to cover compile time evaluation of module name computation
for ``importlib.import_module`` as done by SWIG bindings, allowing
these implicit dependencies to be discovered at compile time without
any help, marking a significant improvement for standalone usage.

-  Annotate type shape for dictionary ``in``/``not in`` nodes, this was
missing unlike in the generic ``in``/``not in`` nodes.

-  Faster processing of "expression only" statement nodes. These are
nodes, where a value is computed, but then not used, it still needs
to be accounted for though, representing the value release.

.. code:: python

   something()  ignores return value, means statement only node

-  Windows: Enabled LTO by default with MinGW64, which makes it produce
much faster results. It now yield faster binaries than MSVC 2019 with
pystone.

-  Windows: Added support for C level PGO (Profile Guided Optimization)
with MSVC and MinGW64, allowing extra speed boosts from the C
compilation on Windows as well.

-  Standalone: Better handling of ``requests.packages`` and
``six.moves``. The old handling could duplicate their code. Now uses
a new mechanism to resolve metapath based importer effects at compile
time.

-  Avoid useless exception checks in our dictionary helpers, as these
could only occur when working with dictionary overloads, which we
know to not be the case.

-  For nodes, have dedicated child mixin classes for nodes with a single
child value and for nodes with a tuple of children, so that these
common kind of nodes operate faster and don't have to check at
runtime what type they are during access.

-  Actually make use of the egg cache. Nuitka was unpacking eggs in
every compilation, but in wheel installs, these can be quite common
and should be faster.

-  Star arguments annotated their type shape, but the methods to check
for dictionary exactly were not affected by this preventing
optimization in some cases.

-  Added ``anti-bloat`` configuration for main programs present in the
modules of the standard library, these can be removed from the
compilation and should lower dependencies detected.

-  Using static libpython with ``pyenv`` automatically. This should give
both smaller (standalone mode) and faster results as is the case when
using this feature..

-  Plugins: Added improvements to the ``anti-bloat`` plugin for
``gevent`` to avoid including its testing framework.

-  Python3.9+: Faster calls into uncompiled functions from compiled code
using newly introduced API of that version.

-  Statically optimize ``importlib.import_module`` calls with constant
args into fixed name imports.

-  Added support for ``sys.version_info`` to be used as a compile time
constant. This should enable many checks to be done at compile time.

-  Added hard import and static optimization for
``typing.TYPE_CHECKING``.

-  Also compute named import lookup through variables, expanding their
use to more cases, e.g. like this:

.. code::

   import sys
   ...
   if sys.version_info.major >= 3:
      ...

-  Also optimize compile time comparisons through variable names if
possible, i.e. the value cannot have changed.

-  Faster calls of uncompiled code with Python3.9 or higher avoiding DLL
call overhead.

Organisational
==============

-  Commercial: There are ``Buy Now`` buttons available now for the
direct purchase of the `Nuitka Commercial </pages/commercial.html>`__
offering. Finally Credit Card, Google Pay, and Apple Pay are all
possible. This is using Stripe. Get in touch with me if you want to
use bank transfer, which is of course still best for me.

-  The main script runners for Python2 have been renamed to ``nuitka2``
and ``nuitka2-run``, which is consistent with what we do for Python3,
and avoids issues where ``bin`` folder ends up in ``sys.path`` and
prevents the loading of ``nuitka`` package.

-  Windows: Added support for Visual Studio 2022 by updating the inline
copy of Scons used for Windows to version 4.3.0, on non Windows, the
other ones will keep being used.

-  Windows: Requiring latest MinGW64 with version 11.2 as released by
winlibs, because this is known to allow LTO, where previous releases
were missing needed binaries.

-  Reject standalone mode usage with Apple Python, as it works only with
the other supported Pythons, avoiding pitfalls in attempting to
distribute it.

-  Move hosting of documentation to Sphinx, added Changelog and some
early parts of API documentation there too. This gives much more
readable results than what we have done so far with Nikola. More
things will move there.

-  User Manual: Add description how to access code attributes in
``nuitka-project`` style options.

-  User Manual: Added commands used to generate performance numbers for
Python.

-  User Manual: List other Python's for which static linking is supposed
to work.

-  Improved help for ``--include-package`` with a hint how to exclude
some of the subpackages.

-  Started using Jinja2 in code templates with a few types, adding basic
infrastructure to do that. This will be expanded in the future.

-  Updated plugin documentation with more recent information.

-  Added Python flavor as detected to the ``--version`` output for
improved bug reports.

-  Linux: Added distribution name to ``--version`` output for improved
bug reports.

-  Always enable the ``gevent`` plugin, we want to achieve this for all
plugins, and this is only a step in that direction.

-  Added project URLs for PyPI, so people looking at it from there have
some immediate places to checkout.

-  Debian: Use common code for included PDF files, which have page
styles and automatic corrections for ``rst2pdf`` applied.

-  Updated to latest ``black``, ``isort``, ``pylint`` versions.

-  The binary names for Python2 changed from ``nuitka`` and
``nuitka-run`` to ``nuitka2`` and ``nuitka2-run``. This harmonizes it
with Python2 and avoids issues, where the ``bin`` folder in
``sys.path`` can cause issues with re-execution of Nuitka finding
those to import.

.. note::

   You ought to be using ``python -m nuitka`` style of calling Nuitka
   anyway, as it gives you best control over what Python is used to
   run Nuitka, you can pick ``python2`` there if you want it to run
   with that, even with full path. Check the relevant section in the
   User Manual too.

-  Added support for Fedora 34 and Fedora 35.

Cleanups
========

-  In a change of mind ``--enable-plugin`` has become the only form to
enable a plugin used in documentation and tests.

-  Massive cleanup of ``numpy`` and Qt binding plugins, e.g.
``pyside2``. Data files and DLLs are now provided through proper
declarative objects rather than copied manually. The handling of
PyQt5 from the plugin should have improved as a side effect.

-  Massive cleanups of all documentation in ReST format. Plenty of
formatting errors were resolved. Many typos were identified and
globally fixed. Spellings e.g. of "Developer Manual" are now enforced
with automatic replacements. Also missing or wrong quotes were turned
to proper methods. Also enforce code language for shell scripts to be
the same everywhere.

-  Removed last usages of ``getPythonFlags()`` and made the function
private, replacing their use with dedicated function to check for
individual flags.

-  Avoid string comparison with ``nuitka.utils.getOS()`` and instead add
accessors that are more readable, e.g. ``nuitka.utils.isMacOS()`` and
put them to use where it makes sense.

-  Replaced usages of string tests in list of python flags specified,
with functions that check for a specific name with a speaking
function name.

-  Added mixin for expressions that have no side effect outside of their
value, providing common method implementation more consistently.

-  Remove code geared to using old PyLint and on Python2, we no longer
use that. Also removed annotations only used for overriding Python2
builtins from Nuitka code.

-  The PDF specific annotations were moved into being applied only in
the PDF building step, avoiding errors for raw PDF directives.

-  Apply Visual Code auto-format to our Yaml files. This is
unfortunately not and automatic formatting yet.

-  Introduce dedicated ``nuitka.utils.Json`` module, as we intend to
expand its usage, e.g. for caching.

-  Replacing remaining usages of ``print`` functions with uses of
``nuitka.Tracing`` instead.

-  Massive cleanup of the ``gevent`` plugin, user proper method to
execute code after module load, rather than source patching without
need. The plugin no longer messes with inclusions that other code
already provides for standalone.

-  Using own helper to update ``sys`` module attributes, to avoid errors
from old C compilers, and also cleaning up using code to not have to
cast on string constants.

-  More consistent naming of plugin classes, and enforce a relationship
of detector class names to the names of detected plugins. The new
naming consistency is now enforced.

Tests
=====

-  Added CPython 3.10 test suite, it needs more work though.

-  Added generated test that exercises dictionary methods in multiple
variations.

-  Test suite names were specified wrongly in a few of them.

Summary
=======

This release is again a huge step forward. It refines on PGO and LTO for
C level to work with all relevant compilers. Internally Python level PGO
is prepared, but only a future release will feature it. With that,
scalability improvements as well as even more performance improvements
will be unlocked.

The amount of optimization added this time is even bigger, some of which
unlocks static optimization of module imports, that previously would
have to be considered implicit. This work will need one extra step,
namely to also trace hard imports on the function level, then this will
be an extremely powerful tool to solve these kinds of issues in the
future. The next release will have this and go even further in this
area.

With the dictionary methods, and some string methods, also a whole new
kind of optimization has been started. These will make working with
``dict`` containers faster, but obviously a lot of ground is to cover
there still, e.g. ``list`` values are a natural target not yet started.
Future releases will progress here.

Type specialization for Python3 has not progressed though, and will have
to be featured in a future releases though.

For scalability, the ``anti-bloat`` work has continued, and this should
be the last release, where this is not on by default. Compiling without
it is something that is immediately noticeable in exploding module
amounts. It is very urgently recommended to enable it for your
compilations.

The support for macOS has been refined, with version information being
possible to add, and adding information to the binary about which OSes
are supported, as well as rejecting Apple Python, which is only a trap
if you want to deploy to other OS versions. More work will be needed to
support ``pyenv`` or even Homebrew there too, for now CPython is still
the recommended platform to use.

This release achieves major compatibility improvements. And of course,
the experimental support for 3.10 is not the least. The next release
will strive to complete the support for it fully, but this should be
usable at least, for now please stay on 3.9 if you can.

***********************

0.6.17

***********************

This release has a focus on performance improvements, while also
polishing plugins and adding many new features.

Bug Fixes
=========

-  Fix, plugins were not catching being used on packages not installed.
Fixed in 0.6.16.2 already.

-  macOS: Fix weaknesses in the ``otool`` parsing to determine DLL
dependency parsing. Fixed in 0.6.16.2 already.

-  Linux: Allow onefile program args with spaces contained to be
properly passed. Fixed in 0.6.16.3 already.

-  Windows: Avoid using less portable C function for ``%PID%``
formatting, which restores compilation on Windows 7 with old
toolchains. Fixed in 0.6.16.3 already.

-  Standalone: Added support for ``fstrings`` package. Fixed in 0.6.16.3
already.

-  Compatibility: Fix, need to import ``.pth`` files after ``site``
module, not before. This was causing crashes on CentOS7 with Python2.
Fixed in 0.6.16.3 already.

-  Compatibility: Fix, when extension modules failed to load, in some
cases the ``ImportError`` was lost to a ``KeyError``. Fixed in
0.6.16.3 already.

-  Fix, linker resource modes ``code`` and ``linker`` were not working
anymore, but are needed with LTO mode at least. Fixed in 0.6.16.3
already.

-  Standalone: Bytecode modules with null bytes in standard library,
typically from disk corruption, were not handled properly. Fixed in
0.6.16.3 already.

-  Fix, failed ``.throw()`` into generators could cause corruption.
Fixed in 0.6.16.4 already.

-  Python2: Fix, the bytecode compilation didn't respect the
``--python-flag=no_asserts`` mode. Fixed in 0.6.16.4 already.

-  Fix, calls were not annotating their arguments as escaped, causing
corruption of mutable in static optimization. Fixed in 0.6.16.5
already.

-  Fix, some sequence objects, e.g. ``numpy.array`` actually implement
in-place add operations that need to be called. Fixed in 0.6.16.5
already.

-  Windows: Fix, onefile binaries were not working after being signed.
This now works.

-  Standalone: Added missing implicit dependency for ``sklearn``.

-  Compatibility: Modules giving ``SyntaxError`` from source were not
properly handled, giving runtime ``ImportError``. Now they are giving
``SyntaxError``.

-  Fix, the LTO mode has issues with ``incbin`` usage on older gcc, so
use ``linker`` mode when it is enabled.

-  Python3: Fix, locals dict codes were not properly checking errors
that the mapping might raise when setting values.

-  Fix, modules named ``entry`` were causing compile time errors in the
C stage.

-  macOS: Never include files from OS private frameworks in standalone
mode.

-  Fix, the python flag ``--python-flag=no_warning`` wasn't working on
all platforms.

-  Compatibility: Fix, the main code of the ``site`` module wasn't
executing, so that its added builtins were not there. Of course, you
ought to use ``--python-flag=no_site`` to not have it in the normal
case.

-  Python2: Added code path to handle edited standard library source
code which then has no valid bytecode file.

-  Anaconda: In module mode, the CondaCC wasn't recognized as form of
gcc.

-  Fix, bytecode modules could shadow compiled modules of the same name.

-  Onefile: Fix, expansion of ``%PID%`` wasn't working properly on
non-Windows, making temp paths less unique. The time stamp is not
necessarily enough.

-  Fix, ``multiprocessing`` error exits from slave processes were not
reporting tracebacks.

-  Standalone: Added ``xcbglintegrations`` to the list of sensible Qt
plugins to include by default, otherwise rendering will be inferior.

-  Standalone: Added ``platformthemes`` to the list of sensible Qt
plugins to include by default, otherwise file dialogs on non-Windows
would be inferior.

-  Fix, created ``.pyi`` files were not ordered deterministically.

-  Standalone: Added support for ``win32file``.

-  Fix, namespace packages were not using runtime values for their
``__path__`` value.

-  Python3.7+: Fix, was leaking ``AttributeError`` exceptions during
name imports.

-  Fix, standard library detection could fail for relative paths.

New Features
============

-  Added experimental support for C level PGO (Profile Guided
Optimization), which runs your program and then uses feedback from
the execution. At this time only gcc is supported, and only C
compiler is collecting feedback. Check the User Manual for a table
with current results.

-  macOS: Added experimental support for creating application bundles.
For these, icons can be specified and console can be disabled. But at
this time, onefile and accelerated mode are not yet usable with it,
only standalone mode works.

-  Plugins: Add support for ``pkg_resources.require`` calls to be
resolved at compile time. These are not working at runtime, but this
avoids the issue very nicely.

-  Plugins: Massive improvements to the ``anti-bloat`` plugin, it can
now make ``numpy``, ``scipy``, ``skimage``, ``pywt``, and
``matplotlib`` use much less packages and has better error handling.

-  Plugins: Added ``anti-bloat`` ability ability to append code to a
module, which might get used in the future by other plugins that need
some sort of post load changes to be applied.

-  Plugins: Added ability to replace code of functions at parse time,
and use this in ``anti-bloat`` plugin to replace functions that do
unnecessary stuff with variants that often just do nothing. T

@pyup-bot
Copy link
Collaborator Author

pyup-bot commented Apr 3, 2022

Closing this in favor of #280

@pyup-bot pyup-bot closed this Apr 3, 2022
@franchg franchg deleted the pyup-update-nuitka-0.5.30-to-0.7.6 branch April 3, 2022 08:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant